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1 Interview procedure tips and tricks 


1.1 Six steps guide 


Now that you know how prepared you need to be, figure out where you are right now. Use CTCI for this. Take a 
couple of questions from each section and solve them using the six steps I mentioned earlier. Keep track of how long 
it takes you to reach an optimal solution for each problem you solve. 


1.1.1 1— Repeat the question in your own words 


As soon as you hear it, repeat it out loud. Do this preferably in your own words to demonstrate your comprehension. 
Remember, your interviewer is there to assist you, so repeating the question aloud will only serve to make their job 
(and yours) easier. 


This will also buy you some time to think about what you’ve been asked and develop good questions or approaches. 
Plus, hearing yourself restate the problem might help you think through it more clearly. 


Clarify the problem statement and explain clearly the working approach. They are looking for thought process. 
1.1.2 2 — Check assumptions 


Many candidates start writing code almost immediately after hearing the question. This is a big mistake. Most coding 
questions have some level of ambiguity built in. Have at least two or three questions ready so that you can confirm you 
have all the necessary detail to solve the problem. You can also write out confirmed assumptions on the whiteboard 
(do it in small print to save space). Questions like “Does input fit in memory?” or “Can I assume input is always 
valid?” are good candidates if you can’t think of any at first. 


Other examples of good clarifying questions to ask: 


e Are there any time or space complexity requirements? 
e How is the input data set specified? In an array? If so, is the array sorted or otherwise structured? 


e What is the data set type and range? Integers, floating points numbers? Does it include positive, negative and 
zero valued inputs? Is the valid range bounded? 


e What data needs to be returned? True/False for success/existence, or the actual data witnessing the property is 
true? 


1.1.3 3— Use real examples 


You’ve got a whiteboarduse it! Draw out an example array, a binary tree, a linked list, etc. Give it real data and 
write out the expected output of a working solution. You should practice coming up with good examples as part of 
your study regime (you do have one, right?). Using a visual example also gives you opportunity to think up more 
questions and your interviewer a chance to correct your assumptions. Don’t forget to keep thinking out loud as you 
work through it. 


If your interviewer provides examples, use those since they probably exist for a reason. This is also how interviewers 
will point you towards problems with your design or implementation. 

1.1.4 4 -— Brainstorm solutions and their time/space complexity 

Stop and think about various approaches. Think about trade offs using Big-O analysis and think out loud. Don’t stop 
with the first solution that comes to mind. Always ask yourself what’s the best you can do. Trust me, we always will! 


Tips: Don’t forget the spacetime tradeoff principle. Wanna go fast? Use more space. Also, hash tables people! And 
learn how to think graphsincluding trees. If all else fails, recursion, backtracking, and memoization are useful tools on 
particularly difficult problems. 


1.1.5 5 — Write working code (no pseudo-code please!) 


You might normally use pseudo-code to design your code, but you don't have time for that in a 45-minute interview. 
Choose your strongest language and turn your thoughts into working code as guickly as possible. This should be the 
easiest part of the interview assuming you've put time into practicing without an IDE (seriously, don't use an IDE 
to practice for the interview unless you're compiling code you've already written out). Make sure to practice using a 
whiteboard or just use pen and paper. 


1.1.6 6 — Test your code, always 


Now turn your brain into a compiler and execute each line of code to ensure you don't have any logical bugs. Use the 
whiteboard to keep track of variable state as you iterate. Also, use the example(s) you created earlier and confirm that 
you get the expected output. Testing is crucial in software engineering, so never make the grave mistake of leaving this 
step out. 


1.1.7 7* — Benchmark yourself (days 3—5) 


Now that you know how prepared you need to be, figure out where you are right now. Use CTCI for this. Take a 
couple of questions from each section and solve them using the six steps I mentioned earlier. Keep track of how long 
it takes you to reach an optimal solution for each problem you solve. 


If and only if youve solved the problem yourself, take a look at the accompanying solution to assess how you did. Did 
you reach the optimal solution or at least progress beyond the naive/brute force answer? How long did it take you? 
Was your code written in the fewest lines possible? 


Do this for every section. When done, you can prioritize the sections that you didnt do so well on up front in your 
practice regimen and leave the other sections for later. You should repeat this exercise just before your interview so 
that you know your weak spots going into the day of the interview. 


1.2 Tips and graph theoretic thinking and modeling 


Whenever someone gives you a problem, think graphs. They are the most fundamental and flexible way of representing 
any kind of a relationship, so it's about a 50-50 shot that any interesting design problem has a graph involved in it. 
Make absolutely sure you can't think of a way to solve it using graphs before moving on to other solution types. This 
tip is important! 


1.3 Videos of feedback and live interviews 


Start with the sourcecheck out google.com/careers for info on how Google hires. Then watch this video (1) from 
Google about what interviewers look for in the interview, and finally check out an example interview (2) featuring real 
Google engineers. 


(1) https: //www. youtube. com/watch?v=ko-KkSmp-Lk 


(2) https: //www. youtube. com/watch?v=XKu_SEDAykw 


TODO What no one tells you about coding interviews (why leetcode doesn’t work) 


TODO My Google Interview — Offer accepted 


1.4 Other links and postings online 


e How should I prepare for my Google software interview if I have one month left? 


Interviewing at Google? Here’s 6 Things You Absolutely Need To Do 


e My Guide To Preparing for the Google Technical Interview 


Coding practice: www.leetcode.com 


e Coding practice: www.interviewcake.com 


1.5 Have a friend interview you 


The friend should ask you a random interview guestion, and you should go write it on the board. You should keep 
going until it is complete, no matter how tired or lazy you feel. Do this as much as you can possibly tolerate. 


Practice a few times with another person, both with someone technical and someone non-technical. Ask them if: 
e You looked and sounded relaxed; 
e You looked like a disciplined problem solver; 


e You kept thinking out loud throughout the entire exercise? 


Sleep and DO NOT be arrogant: You should go in humble, open-minded, and focused. 
1.6 Environment questions for the virtual interview 
e Can I share my screen with a tablet writing device? 


e Can I refer to printed notes I have prepared for the interview? 


2 Programmatic topics in Python3 


2.1 Getter and setter instance methods 
2.2 Math library functionality 
2.3 Built-in list types 


Some common operations on a list include: 


## INITIALIZATION: 

lst = [] 

Ist = [ constValue ] * 1stLength 
lst = strLiteral.split(’’) 


## COMMON OPERATIONS: 

Ist. insert (index, value) 

Ist. remove (value) 

lst .pop (index) 

Ist. find (value) # < 0 if NOT found, else gives the first index 


## SORTING (default algorithm) is in-place (the list itself is modified) and 
## stable (the initial ordering of two equal elements is maintained): 

Ist. sort (key-lambda elt: func(elt), reverse=True|False) 

newList = sorted (Ist, key=lambda xarr: (x[0], x[1]), reverse=True|False) 


2.4 The collections library functionality 
2.5 Implementation of the default dictionary structures 


2.6 Tree implementation with nested dictionaries 


Because Python doesn’t implement trees for us, we'll define them using dictionaries. Each node of the tree will be a 


dictionary that has two keys: 


e The first key is the string “value”, which maps to the value in the node. 


e The second key is the string “children”, which maps to a list of dictionaries, the child nodes. Leaves have empty 


child lists. 


Consider the following binary tree and its associated dictionary-based tree structure: 


t = { "value" : 3, "children" 


[ 
{ "value" : 5, "children" 
[ 
{ "value" : 1, children“: [I Y, 
{ "value" : 4, children“: [] } 
] 
© Q 
> { "value" : 9, children“: [I Y, 
{ "value" : 7, "children" : 
[ 
ey CG (a) { "value" : 8, "children" : [J } 
] 
+ 
] 


2.7 Priority queues (PQ) 

2.8 Unordered sets 

2.9 Multisets 

2.10 Functional programming 


TODO: Know implementation and tradeoffs for use; 
Running times for the internal methods ??? 


2.11 Testing 


Unit tests; design tests for corner or interesting cases of the input data; 


Array Sorting Algorithms 


(gorithm Time Gomplenty | Space Compier 


a 
T 
SIES ESI ESI ERI RAES EASI ESI ERI ESI ESI ESE 
c| ell ell eE Eel D El eE E eE eE eE eE e 
SYS SSS ee l l SS SSS 
S/S |(S]lS|[s s|[s]8][8].8] 818] 8 
% & 
5 


2 
AITAITAITANTAITACANATGI SII Lall Kall Kall Ea] 
s|| illi || ill ill siil s = || E 8 SSIES 
Slll SI SSIS sj e 3 S SSS 
E aja sellele ]| g g eea 
S 8 EEE 
c 818182 
2 [A Alalla alala SSS 
2 irren 
= E Sie KI KII E [| E E 
2 12 sellele] e] 211 211.811. Sl] Si Sis 
ee 
— EI 


Space Complexity 


Clog(n) 
Oln 
Cn+k 


2 
ENESES A 
All all a A 
ISS EEE 
4 4 E te [| > & [ATAANATATATAA TAISI S|] Sil Sila 
m|| aa m SSISISISHISISISISISI SIS AIG 
8 8|| 8 AS AE MEN EEE EMEA ESE 
vol | = || = || = || lll sall s w || = a 8 SS 
Sg 8 S SIS 
oll ell ellellellellellS|i= 11 elle 
wt | | ae | | z 4 Alla — 
©] || © kz 
2 FRESE ERESEEBEE 
ESE SFE 
$ EEE 
sis S 
g 


A AA AMAAN A 

A AllAllAll All AllA 

9 lAl[allallallal Ss c| SHS] Si Sis 
BCS Aaa 
(SSS oS 

DLL] a tf [| i [| et] ee 

T GSS SSS 

© Cll Cci cl co) || T 


lot 
ally A 
Pala ale AE AEA) A 
9 2 9 [880 $ 
FHBIN $ 
E e 8 EE 
AE BBIBEIKEBIBIEIGIEIGIE 
Syl eee |l O 8 
A 
Š 
t 
ri 
2 
A 
E 
td 
ri 
S 


A e 
GL ee 


a 
@(n_Log(n>> 
een logen) 
een logen) 
(n logen) 
Common Data Structure Operations 


La] lalia kal Ea E all Ea] 

a AaYaylayayaya 

IESE 
Ie Ne Ne l 

MEESTE EKI EEE 

E E A a || a 


Best | 
n log(n)) 
O(n Login?) 
n) 
O(n logen)) 


Time Complexity 


3 Data structures and sorting algorithms cheat sheets 
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4 Sorting algorithms — Core material reference 


4.1 Selection sort 


Summary: NON-RECURSIVE & UNSTABLE & IN-PLACE & WC: O(n?) 


In this algorithm, we create two segments of the list one sorted and the other unsorted. We continuously remove the 
smallest element from the unsorted segment of the list and append it to the sorted segment. We dont swap intermediate 
elements. Hence this algorithm sorts the array in the minimum number of swaps. 


A A def selectionSort (array): 
| | | for i in range(len(array)-1): 

min_idx = i 

for idx in range(i + 1, len(array)): 

if arraylidx] < array[min_idx]: 
min_idx = idx 

array[i], array[min_idx] = array[min_idx], array [i] 

return array 


< 


Smallest element moves to left end 


4.2 Insertion sort 


Summary: NON-RECURSIVE & STABLE & IN-PLACE & WC: O(n?) 


Like Selection Sort, in this algorithm, we segment the list into sorted and unsorted parts. Then we iterate over the 
unsorted segment and insert the element from this segment into the correct position in the sorted list. 


def insertionSort (array): 
for i in range(1, len(array)): 
key = array Li! 
j = i-1 
while array[j] > key and j >= 0: 
array[j+1] = array LJ] 
j -=1 
, array[j+i] = key 
return array 


4.3 Heap sort 


Summary: NON-RECURSIVE & UNSTABLE & IN-PLACE & WC: O(n log n) 

We create two segments of the list one sorted and one unsorted. In this, we use the heap data structure to efficiently 
get the max element from the unsorted segment of the list. The heapify method uses recursion to get the max element 
at the top. 


def heapify(array, n, i): 


O largest = i 
1 2 * 1 7 1 


12 2 * 1 2 
if 1 < n and array li] array LI]: 


®© largest = 1 
if r < n and array[largest] < array[r]: 

largest = r 

if largest != i: 


array[i], array[largest] = array[largest], array Li] 
heapify(array, n, largest) 
5 (5) def heapSort(array): 
— n = len(array) 

for i in range(n//2, -1, -1): 

heapify(array, n, i) 
C) for i in range(n-1, 0, -1): 
array Li], array [O] = array[0], array Li] 


O D heapify(array, i, 0) 


return array 


4.4 Merge sort 


Summary: RECURSIVE & STABLE & NEEDS-EXTRA-SPACE & WC: O(nlogn) 


This is a divide and conquer algorithm. In this algorithm, we split a list in half and keep splitting the list by 2 until 
it only has a single element. Then we merge the sorted list. We keep doing this until we get a sorted list with all the 
elements of the unsorted input list. 


def mergeSort (nums) : 

if len(nums)==1: 
return nums 

mid = (len(nums)-1) // 2 
lst1 = mergeSort(nums[:mid+1]) 
lst2 = mergeSort (nums [mid+1:]) 
result = merge(1st1, lst2) 
return result 

def merge(lst1, lst2): 

— split Ist = [] 


x x x x while(i<=len(1st1)-1 and j<=len(1st2)-1): 
if 1stili]<Ist2[j]: 
l1st.append(1st1[i]) 
i+=1 
8 — N d else: 
lst.append(1st2[j]) 
merge j+=1 
if i>len(lst1)-1: 
while(j<=len(1st2)-1): 
lst.append(1st2[j]) 
j+=1 
else: 
while(i<=len(1st1)-1): 
l1st.append(1st1[i]) 
i+=1 
return lst 


4.5 Quick sort 
Summary: RECURSIVE & UNSTABLE & IN-PLACE & WC: Oln log n) 


In this algorithm, we partition the list around a pivot element, sorting values around the pivot. Here, the last element 
from the list as a pivot value. Best performance is achieved when the pivot value splits the list into two almost equal 
halves. 


def quickSort (array): 
if 1en (array) D 1: 
pivot array. pop () 
grtr_lst, equal Ist, smlr_lst = [], [pivot], [J 
for item in array: 
if item == pivot: 
7 equal Ist. append (item) 
elif item > pivot: 


grtr.1st.append (item) 
else: 


smlr. Ist. append (item) 
return (quickSort(smlr_lst) + egual Ist + guickSort(grtr.1st)) 
else: 
return array 


4.5.1 Selection of the pivot elements 


As you can see, Quicksorts efficiency often depends on the pivot selection. If the input array is unsorted, then using 
the first or last element as the pivot will work the same as a random element. But if the input array is sorted or almost 
sorted, using the first or last element as the pivot could lead to a worst-case scenario. Selecting the pivot at random 
makes it more likely Quicksort will select a value closer to the median and finish faster. 


Another option for selecting the pivot is to find the median value of the array and force the algorithm to use it as the 
pivot. This can be done in O(n) time. Although the process is little bit more involved, using the median value as the 
pivot for Quicksort guarantees you will have the best-case Big-O scenario. 


4.6 Counting sort (on non-negative integers) 


Summary: NON-RECURSIVE & STABLE & IN-PLACE-WITH-EXTRA-SPACE & WC: O(n) 


This algorithm does not do a comparison between the elements. We use properties of the integers to sort. We count 
how many times a number has come and store the count in the array where the index is mapped to the keys value. 


def countingSortArray(self, nums: List[int]) -> List[int]: 
i_lower_bound , upper_bound = min(nums), max(nums) 
lower_bound = i_lower_bound 
if i_lower_bound < 0: 
lb = abs(i_lower_bound) 
nums = [item + lb for item in nums] 
lower_bound , upper_bound = min(nums), max(nums) 


counter_nums = [0]*(upper_bound-lower_bound+1) 
for item in nums: 
5 counter_nums[item-lower_bound] += 1 
pos = 0 
for idx, item in enumerate(counter_nums): 
| num = idx + lower_bound 
| 4 | for i in range(item): 
nums [pos] = num 
pos += 1 
if i_lower_bound < 0: 
lb = abs(i_lower_bound) 
nums = [item - lb for item in nums] 
return nums 
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5 Data structures — Core material reference (hash tables) 


5.1 Hash tables 


A hash table organizes data so you can quickly look up values for a given key. 


Pros: 


e Fast lookups: Lookups take O(1) time on average. In many cases, hash tables have average case operation 
running times that are more efficient than search trees or other table-based lookup structures (e.g., arrays). 


e Operations: Insert and deletion are O(1) on average, but O(n) in the worst cases. Space requirement is O(n). 


e Flexible key types: Most data types can be used for keys, as long as they’re hashable. 


Cons: 


Slow (linear-time) worst case lookups: Lookups take O(n) time in the worst case. 


e Unordered: Keys aren’t stored in a special order. If you’re looking for the smallest key, the largest key, or all 
the keys in a range, you'll need to look through every key to find it. 


e Single-directional lookups only: While you can look up the value for a given key in O(1) time, looking up 
the keys for a given value requires looping through the whole dataset — O(n) time. 


e Not cache-friendly: Many hash table implementations use linked lists, which don’t put data next to each other 
in memory. 


5.1.1 Lookup, insert and removal operations 


The advantage of using hashing is that the table address of a record can be directly computed from the key. Hashing 
implies a function h, when applied to a key k, produces a hash M. However, since M could be potentially large, the 
hash result should be mapped to finite entries in the hash table — or slots — several methods can be used to map the 
keys into the size of hash table N. The most common method is the division method, in which modular arithmetic is 


used in computing the slot. 
h(k) = M mod N. 


This is often done in two steps. Namely, compute 
1. Hash = HashFunction(Key); then 


2. Index = Hash (mod HashTableSize). 


The running times of these functions are appealing in the average case, which can be determined by amortized analysis, 
or an average cost of a single operations run m times. 


e Suppose that the hash function h is uniformly distributed so that given any key k, P(h(k) mod N = i) = 4; for 
O<i< N the array indexed that kis initially hashed into. 


e Let the indicator random variable x; be defined such that for k in the key space x;(k) = [h(k) mod N = i]s. 


e Suppose that we want the expected running time of m lookups of the keys {ki}1<i<m. We compute that 


i 1 | 1 
E[Lookup(km)] = N Elxilki) +--+ X En) = — EHC 
* O<i<N O<i<N 1<j<m 
1 n 
00 ->x J, PG -)-— > 
O<i<N 
1<j<m 
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e Then under the assumption that we can resize the array structure of size N to another capacity at least as large 
as 2N while keeping the uniformity property of the hash function in amortized constant time, we can keep the 
last ratio in () bounded within any fixed range we like. That is, the lookup operation can be made O(1) on 
average. 


e Small modifications show that the average running times of the insert and removal operations on the hash table 
run in O(1) expected time, with a worst possible case of O(n) — linear in the number of keys inserted so far. 


5.1.2 Choosing the array size 


A basic requirement is that the hash function should provide a (nearly) uniform distribution of hash values. A 
non-uniform distribution increases the number of collisions and the cost of resolving them. Uniformity is sometimes 
difficult to ensure by design, but may be evaluated empirically using statistical tests, e.g., a Pearson’s chi-squared test 
for discrete uniform distributions. 


The distribution needs to be uniform only for table sizes that occur in the application. In particular, if one uses 
dynamic resizing with exact doubling and halving of the table size, then the hash function needs to be uniform only 
when the size is a power of two. Here the index can be computed as some range of bits of the hash function. On the 
other hand, some hashing algorithms prefer to have the size be a prime number. The modulus operation may provide 
some additional mixing; this is especially useful with a poor hash function. 


5.1.3 Hash table load and rehashing — Design statistics 


The load factor of a hash table is defined to be 

n 

ad 

where n is the number of array entries occupied in the hash table and k is the number of available buckets. The 
perfromance of lookups suffers as a — 1. It is also inefficient to resize the hash table when the load factor is too small 
to justify the operation. A practical ideal value of a might be between 0.6 and 0.75. 


a = HashTableLoadFactor(n, k) 


Repeated insertions cause the number of entries in a hash table to grow, which conseguently increases the load factor; 
to maintain the amortized O(1) performance of the lookup and insertion operations, a hash table is dynamically resized 
and the items of the tables are rehashed into the buckets of the new hash table,[13] since the items cannot be copied 
over as varying table sizes results in different hash value due to modulo operation. Resizing may be performed on hash 
tables with fewer entries compared to its size to avoid excessive memory usage. 


If the design requires that the array size be prime. We need to find the next prime number that is greater than 2N + 1 
where N is the current (prime-valued) size of the array. Finding the next such prime can be performed in O (N 2) 


time (see this link, in section finding the next prime, for example). 
5.1.4 Collision resolution: Open addressing and separate chaining 


This strategy is also called two-choice hashing. The first part is to compute the hash function on a key which resolves 
the initial array index of that key. The second step is to perform collision resolution when distinct keys still collide at 
the same array index. Common collision resolution strategies include open hashing (or open addressing) and separate 
chaining (or just chaining). 


Discussion of separate chaining: The basic API for operations is 


ChainedHashlnsert (HT, x): 

insert x at the head of the linked list HT[h(x.key)] 
ChainedHashSearch (HT, k): 

search for element with key k in linked list HT[h(k)] 
ChainedHashDelete (HT, x): 

delete x from the linked list located at HT[h(x.key)] 


If the keys of the elements are ordered, it’s efficient to insert the item by maintaining the order when the key is 
comparable either numerically or lexically, thus resulting in faster insertions and unsuccessful searches. However, the 
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standard method of using a linked list is not cache-conscious since there is little spatial localitylocality of referencesince 
the nodes of the list are scattered across the memory, hence doesn’t make efficient use of CPU cache. In cache-conscious 
variants, a dynamic array found to be more cache-friendly is used in the place where a linked list or self-balancing 
binary search trees is usually deployed for collision resolution through separate chaining, since the contiguous allocation 
patten of the array could be exploited by hardware-cache prefetcherssuch as translation lookaside buffer-resulting in 
reduced access time and memory consumption. 


keys buckets entries 


Discussion of open addressing: 


All entry records are stored in the bucket array itself. When a new entry has to be inserted, the buckets are examined, 
starting with the hashed-to slot and proceeding in some probe seguence, until an unoccupied slot is found. When 
searching for an entry, the buckets are scanned in the same seguence, until either the target record is found, or an 
unused array slot is found, which indicates that there is no such key in the table. Well-known probe seguences include: 
linear probing, guadratic (polynomial) probing and double hashing where a second hash function is used to resolve the 
next available slot in the array. 


keys buckets 


Discussion of coalesced hashing: 


Coalesced hashing is a hybrid of both separate chaining and open addressing in which the buckets or nodes link within 
the table. The algorithm is ideally suited for fixed memory allocation. This method is similar to separate chaining in 
that it gives a pointer parameter to each bucket that tracks the location of the next collision node, creating a traceable 
chain of each node that collided at the original index. 


o ofu — 

— Oe 

1 gel = 

2 4 gur Y 

3 clq = 
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5.1.5 Linked hash sets 


The purpose of the linked hash set object, as we have seen, is to keep track of the entry order of elements while retaining 
its O(1) average times for the lookup-based operations. The way to achieve this effect is to maintain, in addition to 
the hash table, a (doubly) linked list to hold the actual elements. Hence,we can use this structure variant to not only 
get the benefits of a hash table, but also provide an iterator to the keys preserving the original order (or something 
else) in which the keys have been inserted into the table. 


element index 
"music 2 
"beer" 5 
"afterlife" 9 
"wisdom" 4 
"politics" 10 
"theater" 2 
"schools" 1 
"painting" 5 
"fear" 3 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

first 
10 


5.2 Hash sets 
A hash set is an implementation of the unordered set interface which does not allow duplicate value. 
5.3 Hash maps 


A hash map is an implementation of the map interface which maps a key to a value (lookups stored via a hash table 
mechanism). Note that duplicate keys are forbidden in a map instance. 
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5.4 Dictionary trees — Trie-trees — or digital trees — or prefix trees 


a a 
Structure Node 
n Children Node[AlphabetSize] 
t IsTerminal Boolean 
d Value DataType 
a 


## WORST-CASE: O(m) where m is thee length of the key to search for 
## AVERAGE-CASE: O(log_m(n)) where n is the total number of keys and m 
#H is the alphabet size 
TrieFind(x, key) 
for i in range(0, len(key) + 1): 
if x.Children[key[i]] == None: 
return False 
x = x.Children[key[i]] 
return x.Value 
## WORST-CASE (TIME): O(|wl * n) where w is the search key and n is the 
titt number of keys in the tree 
## WORST-CASE (SPACE): O(|wl + n) L. . . I 
TrieInsert(x, key, value) 
for i in range(0, len(key) + 1): 
if x.Children[key[i]] == None: 
x.Children[key[i]] = Node() 
x KX. Children [key Li]! 
X. Value := value 
X. Is Terminal = True 
TrieDelete (x, key) 
if key == None: 
if x.IsTerminal: 
x.IsTerminal = False 
x.Value = None 
return None 
x.Children[key[0]] = TrieDelete(x.Children[key[0]], key[1:]) 
return x 


5.4.1 Trie-trees as a replacement for hash tables 


e Searching for a node with an associated key of size m has the complexity of O(m), whereas an imperfect hash 
function may have numerous colliding keys, and the worst-case lookup speed of such a table would be O(N), 
where N denotes the total number of nodes within the table. 


e Tries do not need a hash function for the operation, unlike a hash table; there are also no collisions of different 
keys in a trie. 


e Buckets in a trie, which are analogous to hash table buckets that store key collisions, are necessary only if a single 
key is associated with more than one value. 
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e String keys within the trie can be sorted using a predetermined alphabetical ordering. 


However, tries are less efficient than a hash table when the data is directly accessed on a secondary storage device such 
as a hard disk drive that has higher random access time than the main memory. Tries are also disadvantageous when 
the key value cannot be easily represented as string, such as floating point numbers. 


5.4.2 Compressed versions — see radix trees below 


A radiz tree, also known as a compressed trie, is a space-optimized variant of a trie in which nodes with only one child 
get merged with its parents; elimination of branches of the nodes with a single child results in better in both space 
and time metrics. This often works best under conditions where the trie remains static and set of keys stored are very 
sparse within their representation space. 


5.4.3 External memory tries 


Several trie variants are suitable for maintaining sets of strings in external memory, including suffix trees. A combination 
of trie and B-tree, called the B-trie has also been suggested for this task; compared to suffix trees, they are limited in 
the supported operations but also more compact while performing update operations faster. 


5.4.4 Applications 


e Lexicographical sorting: Lexicographic sorting of a set of string keys can be implemented by building a trie 
for the given keys and traversing the tree in pre-order fashion; this is also a form of radix sort. Tries are also 
fundamental data structures for burstsort, which is notable for being the fastest string sorting algorithm as of 
2007, which is accompanied for its efficient use of CPU cache. 


e Web search engines: A specialized kind of trie called a compressed trie, is used in web search engines for 
storing the indexes — a collection of all searchable words. Each terminal node is associated with a list of URLs 
— called occurrence list — to pages that match the keyword. The trie is stored in the main memory, whereas the 
occurrence is kept in an external storage, frequently in large clusters. 


e Internet routing: Compressed variants of tries, such as databases for managing Forwarding Information Base 
(FIB), are used in storing IP address prefixes within routers and bridges for prefix-based lookup to resolve 
mask-based operations in IP routing. 


5.5 Hash chains 


Most cryptographic hash functions are designed to take a string of any length as input and produce a fixed-length hash 
value. The security level of a cryptographic hash function has been defined using the following properties: 


e Pre-image resistance: Given a hash value h, it should be difficult to find any message m such that h = hash(m). 
This concept is related to that of a one-way function. 


e Second pre-image resistance: Given an input mi, it should be difficult to find a different input ma such that 
hash(m1) = hash(m2). This property is sometimes referred to as weak collision resistance. 


e Collision resistance: It should be difficult to find two different messages mı and ma such that hash(m;1) = 
hash(m2). Such a pair is called a cryptographic hash collision. This property is sometimes referred to as strong 
collision resistance. It requires a hash value at least twice as long as that required for pre-image resistance; 
otherwise collisions may be found by a birthday attack. 


A hash chain is a successive application of a cryptographic hash function h to a string x. For example, h(h(h(x))) = 
h3(a) is a hash chain of length 3. Binary hash chains are commonly used in association with a hash tree. A binary hash 
chain takes two hash values as inputs, concatenates them and applies a hash function to the result, thereby producing 
a third hash value (cf. Merkle trees). 


16 


Xroot="%14!%5g) Xroot=h(x44!X5g) 


xsg=h(XsglX7g) x44 No 
X58 
x42=h(x41x9) X7g=h(x7lxg) *12 334 


J CN. LY, 4. N. 


RLR {101} 
The RLR (101) order bits are necessary to complete the hash chain above. 


IA -H 


5.6 Unordered sets 


A (unordered) set is like a hash map except it only stores keys, without values. Usually, we’re interested in whether 
something is in a set or not. Sets are usually implemented very similarly to hash maps — using hashing to index into 
an array — but they don’t have to worry about storing values alongside keys. These are implemented using hash maps 
— each member of the set is a key in the hash map with a dummy value that gets ignored. 


5.7 Bloom filters 


TODO: See https: //www.interviewcake.com/concept/python/bloom-filter? 
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6 Data structures — Core material reference (linked lists, stacks, gueues and 
heaps) 

6.1 Doubly versus singly linked lists — Usage and operation running times 

6.2 Dynamic arrays (and expected constant time operations with resizing) 

6.3 Skip lists 

6.4 Stacks 

6.5 Queues (generic container) 

6.5.1 Binomial queues 


Other types of common — See Java API ??? 


TODO: See https: //en.wikipedia. org/wiki/Priority_queue 
6.6 Heaps 
TODO: Min-heaps ??? 


6.6.1 Priority queues 
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7 Data structures — Core material reference (trees and traversals) 


7.1 Binary and n-ary trees 


Examples of usage: 


e Directory structures in a filesystem; 
e Company organizational heirarchy; 


e Sports tournament brackets. 


7.2 Tree traversals 

7.2.1 BFS and DFS 

7.2.2 In-order, pre-order and post-order traversals 
7.2.3 Sorting (generic trees) 


The length of the maximal path through the tree on n nodes is n+ 1. A generic sorting algorithm on any tree (without 
knowing more about its underlying structure) is therefore O(n) in the worst case. 


7.3 Binary search trees (BSTs) 


TODO: See CMU site slides {= page 21; 


7.4 Balanced (binary) trees 

7.4.1 Red-Black trees 

7.4.2 Splay trees 

7.4.3 AVL trees 

7.4.4 B-trees ??? 

TODO: See https: //en.wikipedia.org/wiki/B-tree 

7.5 Radix trees (Patricia tries) 

TODO: See https: //en.wikipedia.org/wiki/Radix_tree 
7.6 -D trees 

7.7 Quad trees 


19 


8 Data structures — Core material reference (graphs) 
8.1 Implementations and representations in code 

8.1.1 Objects and pointers 

8.1.2 Matrix-based storage 

8.1.3 Adjacency lists 

8.2 BFS and DFS 

8.3 Dijkstra’s algorithm (TODO — Purpose) 


TODO: See https: // en. vikipedi a. org/wiki/Dijkstra/,27s_algorithm 
TODO: For running time charts by graph type case, see https: // en. vi ki pedi a. org/wiki/Shortest_path_problem# 
Single-source_shortest_paths 


8.4 Prim’s algorithm (TODO — Purpose) 

TODO: See https://en.wikipedia.org/wiki/Prim%27s algorithm 
8.5 Kruskal’s algorithm (TODO — Purpose) 

TODO: See https://en.wikipedia.org/wiki/Kruskal%275s. algorithm 
8.6 Bellman-Ford algorithm (TODO — Purpose) 

TODO: See https://en.wikipedia.org/wiki/BellmanE2%80%93Ford. algorithm 
8.7 Floyd-Warshall algorithm (all pairs shortest paths) 

8.8 A algorithm 

TODO: See https: // en. vikipedi a. org/wiki /A search algorithm 
8.9 General purpose algorithms 

8.9.1 Distance 

8.9.2 Search problems 

8.9.3 Graph connectivity problems 

8.9.4 Cycle- detection algorithms 

8.10 Topological sort 


8.11 Probabilistic methods and expected running times and optimal solution guarantees 
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9 Recursion, backtracking and memoization — Core material reference 


9.1 Dynamic programming — Top-down versus bottom-up approaches 


Top-down approach: This is the direct fall-out of the recursive formulation of any problem. If the solution to any 
problem can be formulated recursively using the solution to its sub-problems, and if its sub-problems are overlapping, 
then one can easily memoize or store the solutions to the sub-problems in a table. Whenever we attempt to solve a 
new sub-problem, we first check the table to see if it is already solved. If a solution has been recorded, we can use it 
directly, otherwise we solve the sub-problem and add its solution to the table. 


Bottom-up approach: Once we formulate the solution to a problem recursively as in terms of its sub-problems, we 
can try reformulating the problem in a bottom-up fashion: try solving the sub-problems first and use their solutions 
to build-on and arrive at solutions to bigger sub-problems. This is also usually done in a tabular form by iteratively 
generating solutions to bigger and bigger sub-problems by using the solutions to small sub-problems. For example, if 
we already know the values of Tai and F49, we can directly calculate the value of F42. 


9.2 Backtracking — Includes general procedure pseudocode 


Backtracking is a general algorithm for finding solutions to some computational problems, notably constraint satisfac- 
tion problems, that incrementally builds candidates to the solutions, and abandons a candidate (“backtracks” ) as soon 
as it determines that the candidate cannot possibly be completed to a valid solution. 


9.2.1 Description of the method 


The backtracking algorithm enumerates a set of partial candidates that, in principle, could be completed in various 
ways to give all the possible solutions to the given problem. The completion is done incrementally, by a sequence of 
candidate extension steps. 


Conceptually, the partial candidates are represented as the nodes of a tree structure, the potential search tree. Each 
partial candidate is the parent of the candidates that differ from it by a single extension step; the leaves of the tree 
are the partial candidates that cannot be extended any further. 


The backtracking algorithm traverses this search tree recursively, from the root down, in depth-first order. At each 
node c, the algorithm checks whether c can be completed to a valid solution. If it cannot, the whole sub-tree rooted 
at cis skipped (pruned). Otherwise, the algorithm (1) checks whether c itself is a valid solution, and if so reports it to 
the user; and (2) recursively enumerates all sub-trees of c. The two tests and the children of each node are defined by 
user-given procedures. 


Therefore, the actual search tree that is traversed by the algorithm is only a part of the potential tree. The total cost 
of the algorithm is the number of nodes of the actual tree times the cost of obtaining and processing each node. This 
fact should be considered when choosing the potential search tree and implementing the pruning test. 


9.2.2 Pseudocode implementation (generic algorithm) 


In order to apply backtracking to a specific class of problems, one must provide the data P for the particular instance 
of the problem that is to be solved, and six procedural parameters. These procedures should take the instance data P 
as a parameter and should do the following: 


1. root(P): return the partial candidate at the root of the search tree. 

2. reject(P,c): return true only if the partial candidate c is not worth completing. 

3. accept (P, c): return true if c is a solution of P, and false otherwise. 

4. first(P,c): generate the first extension of candidate c. 

5. next(P,s): generate the next alternative extension of a candidate, after the extension s. 


6. output(P,c): use the solution c of P, as appropriate to the application. 
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The backtracking algorithm reduces the problem to the call backtrack(root(P)), where backtrack is the following 
recursive procedure: 


procedure backtrack(c) is 
if reject(P, c) then return 
if accept(P, c) then output(P, c) 
s <- first(P, c) 
while s != NULL do 
backtrack(s) 
s <- next(P, s) 


Early stopping variants: The pseudo-code above will call output for all candidates that are a solution to the given 
instance P. The algorithm can be modified to stop after finding the first solution, or a specified number of solutions; 
or after testing a specified number of partial candidates, or after spending a given amount of CPU time. 


9.2.3 Examples 
e Constraint satisfaction of a boolean-valued function on m integer variables; 


e See blog article: Backtracking, Memoization & Dynamic Programming! 


9.3 Memoization 
9.4 Dynamic programming 
9.5 Greedy algorithms 


TODO: See https: // en. vikipedi a. org/wiki/Greedy_algorithm#Submodular_functions 
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10 Other algorithmic type preparation 


10.1 Divide and conquer 

10.2 Prime factorizations 

10.3 Huffmann coding 

10.4 Error correcting codes 

10.5 Hamming codes 

10.6 Variable-length codes 

TODO: See https://en.wikipedia.org/wiki/Variable-length. code 

10.7 Best known algorithms for matrix operations 

10.8 Matrix decompositions — SVD, LU, QR, Cholesky (LDL), Householder, etc. 
10.9 Least squares, linear least squares and best fit modeling problems 


Let X be an x p matrix, 6 be a px 1 column vector and y an x column vector. In general, this is an over-determined 
system of n linear equations in p unknown coefficients (the rows of 6) satisfying 


> Xijbj = Yi, fori=1,2,...,n. 


1<j<p 


This system can be expressed in matrix form as X8 = y. Since such a system usually cannot be solved exactly, we 
seek to find the coefficients (6;)1<i<p that are the “best fit”, here with respect to the following quadratic minimization 


problem: 
2 


Ê = argming S(6), where $(8):= N = |y — XBI)?. 


l<i<n 


yvi- >) X55 


1<j<p 


This particular form of the minimization problem has a unigue solution whenever the p columns of X are linearly 
independent: 

Ê = (XTX) XTy = b (XTX) XI. 
10.10 Sequence alignment — Dynamic programming application 


The problem can be stated naturally as a recursion, a sequence A is optimally edited into a sequence B by either: 


1. Inserting the first character of B, and performing an optimal alignment of A and the tail of B; 
2. Deleting the first character of A, and performing the optimal alignment of the tail of A and B; 


3. Replacing the first character of A with the first character of B, and performing optimal alignments of the tails 
of A and B. 


The partial alignments can be tabulated in a matrix, where cell (7,7) contains the cost of the optimal alignment of 
A[1..i] to B[1..5]. The cost in cell (i, j) can be calculated by adding the cost of the relevant operations to the cost of 
its neighboring cells, and selecting the optimum. 


10.11 Locality of reference 


Locality of reference, also known as the principle of locality, is the tendency of a processor to access the same set of 
memory locations repetitively over a short period of time. There are two basic types of reference locality — temporal 
and spatial locality. Temporal locality refers to the reuse of specific data and/or resources within a relatively small time 
duration. Spatial locality (also termed data locality) refers to the use of data elements within relatively close storage 
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locations. Seguential locality, a special case of spatial locality, occurs when data elements are arranged and accessed 
linearly, such as traversing the elements in a one-dimensional array. 
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11 NP-completeness and standard problems 


11.1 Precise definition of the NP-complete problem class 
11.2 Traveling salesman problem (TSP) 


The travelling salesman problem (also called the travelling salesperson problem or TSP) asks the following question: 
Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each 
city exactly once and returns to the origin city? 

It is an NP-hard problem in combinatorial optimization. 


TODO: See https: //en.wikipedia. org/wiki/Travelling_salesman_problem 
11.3 Knapsack problem 

TODO: See https://en.wikipedia.org/wiki/Knapsack_problem 

11.4 Subset sum problem (SSP) 

TODO: See https://en.wikipedia.org/wiki/Subset_sum_problem 

11.5 Partition problem 


The partition problem, or number partitioning, is the task of deciding whether a given multiset S of positive integers 
can be partitioned into two subsets Si and S2 such that the sum of the numbers in 5; equals the sum of the numbers 
in S2. This problem corresponds to the SSP where all inputs are positive, and the target sum is exactly half of all 
inputs: T = 30 +--+ dn). 


11.6 Independent sets (graph theory) 

TODO: See https://en.wikipedia.org/wiki/Independent_set_(graph_theory) 
11.7 Set cover problem (SCP) 

TODO: See https://en.wikipedia.org/wiki/Set_cover_problem 

11.8 Bin packing problem (BPP) 

TODO: See https://en.wikipedia.org/wiki/Bin_packing_problem 

11.9 Multiway number partitioning 

TODO: See https://en.wikipedia.org/wiki/Multiway_number_partitioning 
11.10 Randomized algorithmic approaches 

11.10.1 The Monte-Carlo method 

TODO: See https://en.wikipedia.org/wiki/Monte_Carlo_method 


11.10.2 Graph coloring type problems ??? 
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12 Operating systems and systems programming mechanics 


12.1 Systems design (SWE specs) 


Questions are used to assess a candidate’s ability to combine knowledge, theory, experience and judgement toward 
solving a real-world engineering problem. In other words, can a candidate ”design policies, processes, procedures, 
methods, tests, and/or components from ground up. 


Sample topics include: Features sets, interfaces, class hierarchies, distributed systems, designing a system under 
certain constraints, simplicity, limitations, robustness and tradeoffs. You should also have an understanding of how the 
internet actually works and be familiar with the various pieces (routers, domain name servers, load balancers, firewalls, 
etc.). Other topics include: traverse a graphs, run-time complexity of graphs, distributed hash table system, resource 
estimation with real systems, big product design picture, translation of an abstract problem to a system, API discus- 
sions, binary trees, cache, mapreduce, for loop problems, index, reverse link-list, compilers, memory cache, networks 
and also common topics. 


12.2 Operating systems (SWE specs and advice from online) 


SWE statement: You should understand processes, threads, concurrency issues, locks, mutexes, semaphores, mon- 
itors and how they all work. Understand deadlock, livelock and how to avoid them. Know what resources a process 
needs and a thread needs (also: resourcem allocation for processes and threads). Understand how context switching 
works, how it’s initiated by the operating system and underlying hardware. Know a little about scheduling. We are 
rapidly moving towards multi-core, so know the fundamentals of ” modern” concurrency constructs. 


BlogSpot tips: This is just a plug, from me, for you to know about processes, threads and concurrency issues. A 
lot of interviewers ask about that stuff, and it’s pretty fundamental, so you should know it. Know about locks and 
mutexes and semaphores and monitors and how they work. Know about deadlock and livelock and how to avoid them. 
Know what resources a processes needs, and a thread needs, and how context switching works, and how it’s initiated by 
the operating system and underlying hardware. Know a little about scheduling. The world is rapidly moving towards 
multi-core, and you'll be a dinosaur in a real hurry if you don’t understand the fundamentals of “modern” (which is 
to say, “kinda broken”) concurrency constructs. 

Source: http://steve-yegge. blogspot .com/2008/03/get-that- job-at-google. html 


12.3 Types of design tradeoffs — Examples 
12.3.1 Lookup tables vs. recalculation 


A common situation is an algorithm involving a lookup table: an implementation can include the entire table, which 
reduces computing time, but increases the amount of memory needed, or it can compute table entries as needed, 
increasing computing time, but reducing memory requirements. 


12.3.2 Compressed vs. uncompressed data 


A spacetime tradeoff can be applied to the problem of data storage. If data is stored uncompressed, it takes more space 
but access takes less time than if the data were stored compressed (since compressing the data reduces the amount 
of space it takes, but it takes time to run the decompression algorithm). Depending on the particular instance of the 
problem, either way is practical. There are also rare instances where it is possible to directly work with compressed 
data, such as in the case of compressed bitmap indices, where it is faster to work with compression than without 
compression. 


12.3.3 Re-rendering vs. stored images 


Storing only the SVG source of a vector image and rendering it as a bitmap image every time the page is requested 
would be trading time for space; more time used, but less space. Rendering the image when the page is changed and 
storing the rendered images would be trading space for time; more space used, but less time. This technique is more 
generally known as caching. 
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12.3.4 Smaller code vs. loop unrolling 


Larger code size can be traded for higher program speed when applying loop unrolling. This technique makes the code 
longer for each iteration of a loop, but saves the computation time required for jumping back to the beginning of the 
loop at the end of each iteration. 


12.4 Systems — The basics of systems programming 
12.5 Scheduling algorithms 

12.6 Internet and networking basics 

12.6.1 Routers 

12.6.2 DNS and hierarchy 

12.6.3 Load balancers 

12.6.4 Firewalls 

12.7 Distributed systems 

12.8 Flow networks (graphs) 


TODO: See https://en.wikipedia.org/wiki/Flow_network 
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13 OOP fundamentals 
13.1 Abstraction 

13.2 Inheritance 

13.3 Cohesion 

13.4 Coupling 
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14 Functional programming constructs 


14.1 Map- Reduce 
14.2 Reap- So w 
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15 Architecture and compilers 

15.1 Compiling and linking 

15.2 Cache and memory cache 

TODO: See https://en.wikipedia.org/wiki/CPV. cache 
15.3 LRU cache 


TODO: See https://www.interviewcake.com/concept/python/lru-cache? 
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16 Languages and formal grammars 
16.1 Grammars 
16.2 Lexers and tokenizers 


16.3 Parsers and syntax flow rules 
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17 Basic discrete math and statistics 


17.1 Counting — Four types chart 
17.2 Probability review 
17.2.1 Independence 


Two events A,B are independent if P(A n B) = P(A) x P(B). A collection of events {A;} are mutually, or pairwise, 
independent provided that A; and A; are independent whenever i + j. 


17.2.2 Conditional probability and Bayes theorem 
The conditional probability of the event A given the event B is defined to be 


P(An B) 


P(A|B) = PB) 


where P(A|B) = P(A) whenever A, B are independent. 


Suppose that we have a collection of events {A;}. Then Baye’s theorem states that (cf. posterior probabilities) 


P(BJA)P(A) 


P(A1B) = pp, 


, if P(B) 40. 


Often this can be combined with the next identity when B is the union of the pairwise disjoint events {B;};>1: 


P(B) = >, P(A|B;)P(Bi). 


izl 


17.2.3 Conditional distributions and conditional expectation 


We have that E[Y |Y] = Y and EE[X|Y] = E[X]. Precisely, the conditional distribution of X given Y is defined to be 


P(X=xz^AY =y) 


Discrete RVs EIXIY = y| = ; 
1 (0 0) 
(Continuous RVs) =a | x(, da, 
fy (y) —00 


where fy is the marginal distribution of Y and fx y is the joint distribution of X and Y. We also have that for a 
continuous random variable X with PDF fx, 


P(A) = [ P(A|X = 2) fx(a)dz. 


17.2.4 Markov's inequality and formulas for first moments 


If X > 0, then 


P(X > a) < —1 55. 


The first moment method is based on the property that if X > 0, then E[X] = 0 «=> X =0 almost surely. 
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17.2.5 Chebyshev’s inequality and the second moment method 


We have the following variants of Chebyshev's inequality for Var(X) = o? < +00 where u := E[X] and the parameters 
k,e > O: 


e P(|z— u| > ko) < $. 


e (% —pl < e) 1 f. 


The second moment method is based on the property that if X > 0 and X has finite variance, then 


EX] 
EIN 


N 


P(X >0) > 


— 


17.2.6 Laws of large numbers (weak and strong) 


The WLLN can be applied in some cases, i.e., when Lebesgue integration predicts no first moment exists for a random 
variable, where the SLLN cannot be applied. The SLLN states that for X, := 4 (Xl +--+ Xn) with {Xj}is1 a 
sequence of IID random variables that are integrable so that their expected value exists and is finite, we have 


X. Os ELXY]. 


The variances Var(X;) need not be finite. The WLLN states that for any fixed € > 0, 
lim P (Xn — E[X1]| < €) = 1. 


N=—>00 


Note that the WLLN allows that |X, — E[X1]| > e infinitely often, though at increasingly infrequent intervals as 
n — œ. In contrast, when the SLLN applies, for any fixed € > 0 at all sufficiently large n > N (e), we must have that 
[X. — EDX]| < €. 


17.2.7 Central limit theorem 


Let (Xi ii be a sequence of IID random variables with first moment u and variance g? < +00. We have the convergence 
in distribution to 


Jim yn (== £) ~ N (1,00). 
17.2.8 Borel-Cantelli lemma — TODO 
17.2.9 Caratheodory criterion 
(B) = (Bn A) + u(B\A) 
17.2.10 Monotone convergence theorem for expectation 
17.3 Basic statistical calculations 
17.3.1 Types of averages and inequalities 
We have the following types of means for a collection (dien of n numbers: 
e Arithmetic mean: TODO 
e Geometric mean: TODO 
e Harmonic mean: TODO 


e Root mean square (RMS) average: TODO 


The AGM inequality states that TODO ... 
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17.3.2 Variance and covariance properties 


The variance of a random variable X is defined to be Var(X) := E[X?] — E[X]?. We have that Var(X + b) = Var(X) 
for any real b and that for all a € R: Var(aX) = a? Var(X). Moreover, if {Xi}i<i<n are independent random variables, 
then 


Var(ayX1 +--+ + an Xn) = a? Var(X1) +--+ + a2 Var(X,,). 


If we have a sample of numbers {z;}1<;<N with arithmetic (sample) mean 


then the sample variance is defined to be 
1 1 1 
2 2 2 2 2 
o“ := — x >; (27 — u) - (3> ) ot = sa 3 (epee): 
N 1<i<N Li 1<i<N on 1<i,j<N 
The standard deviation of the collection of numbers is given by o — the square root of the variance. 


For two random variables X, Y we define their covariance to be Cov(X, Y) = E[(X —E[X])(Y —E[Y])]. The covariance 
is linear in each argument. We have that 


Var(14) = P(A)(1 — P(A). 


Similarly, the correlation between X and Y is defined to be Corr(X,Y) = Cov(X,Y)/oxoy. For all real a,c * O, 
Corr(a X + b, cY + d) = Corr(X,Y). We also have 


Cov(14,1g) = P(An B) - P(A)P(B). 
17.3.3 Law of large numbers 


17.4 Cauchy-Schwarz and I inequalities 


CARIE 
[fro < fire five. 


% < flle x ill 


We have that 


and more generally that 


fi+i=l <=> p +q = pg, then 


Minkowski’s inequality provides that 
lf + gllp < II Fllp + IIgllp- 
Also, for any measure u and a O, we have that 


p 
nila: Ifa > a) (Hale) 


a 
17.5 Inclusion-exclusion and variants 
17.6 Miscellaneous approximations and inequalities — See two sections of green notebook 
17.6.1 Jensen’s inequality 
Recall that a function F: R — R is called convex if for all te [0,1] and x,y € R, we have that 
f(t + (1— ty) <tf(x) + 1 — t) f(y). 
If X is a random variable and f is convex, then 


fE[X]) < ELF). 
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17.6.2 Average value inegualities for monotone functions 


If f,g : [a,b] > R are monotone functions of the same monotonicity, then 


— x [ tosa) > 7 x Schar x [oa 


The order of the inequality is reversed if f and g have opposite monotonicity on [a,b]. 
17.7 Stochastic processes (and Markov chains) 
TODO: See https: //en.wikipedia. org/wiki/Markov_chain#Formal_definition 


17.8 Martingales 
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